Generating effects in a webcam application

ABSTRACT

A system and a method for generating effects in a webcam application are provided. The method includes identifying a first object and a second object in a video image. The method also includes adding a first user-created object to the video image to create an altered video image and adding a second user-created object to the altered video image to further alter the altered video image. Other steps included are associating the second user-created object with the second object; identifying a movement of the second object; and moving the second user-created object in the altered video image in accordance with the association of the second user-created object with the second object. The first object is a static object, and the first user-created object is manually movable. The movement of the second user-created object in association with the second object is independent of a movement of the first user-created object.

TECHNICAL FIELD

The present invention is generally related to video and audio technologyand, more particularly, is related to a system and method for generatingeffects in a webcam application.

BACKGROUND

In recent years, a growing number of personal computers and interactivetelevision systems are equipped with digital video cameras. Thesecameras may be configured as a web camera or “webcam.” The webcamcaptures a continual stream of video images and broadcasts the images onthe monitor of a personal computer. Such cameras may be used for atleast one-way video communication, two-way video communication(videoconferencing, chatting, etc), or broadcast video communication.The communication may be displayed on the monitor or can be transmittedthrough a network such as a local area network (LAN) or the Internet.

SUMMARY

Embodiments of the present invention provide a system or method forgenerating effects for a webcam application. In one embodiment, a methodfor generating effects for a webcam application is provided. In thisregard, one embodiment of such a method, among others, can be broadlysummarized by the following steps: identifying a first object in a videoimage; identifying a second object in the video image; adding a firstuser-created object to the video image to create an altered video image;adding a second user-created object to the altered video image tofurther alter the altered video image; associating the seconduser-created object with the second object; identifying a movement ofthe second object; moving the second user-created object in the alteredvideo image in accordance with the association of the seconduser-created object with the second object. The first object may be astatic object, and the first user-created object may be manuallymovable. The movement of the second user-created object in associationwith the second object may be independent of a movement of the firstuser-created object.

In another embodiment, a method for generating effects for a webcamapplication can be broadly summarized by the following steps:identifying a first object and a second object in a video image; addinga first user-created object to the first object to create an alteredvideo image; adding a second user-created object to the second object tofurther alter the altered video image; associating the firstuser-created object with the first object; associating the seconduser-created object with the second object; identifying a movement ofthe first object and a movement of the second object, respectively; andmoving the first user-created object in accordance with the associationof the first user-created object with the first object.

Briefly described, in architecture, one embodiment of the system, amongothers, can be implemented as follows. A system for generating effectsfor a webcam application may comprise an identification module foridentifying a first object in a video image; an addition module foradding at least one user-created object to the video image to create analtered video image; an association module associating the at least oneuser-created object with the first object; a motion detection module foridentifying a movement of the first object; a movement module for movingthe at least one user-created object in accordance with the associationof the at least one user-created object with the first object in thealtered video image; and a display module for displaying the alteredvideo image.

Other systems, methods, features, and advantages of the presentinvention will be or become apparent to one with skill in the art uponexamination of the following drawings and detailed description. It isintended that all such additional systems, methods, features, andadvantages be included within this description, be within the scope ofthe present invention, and be protected by the accompanying claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the invention can be better understood with reference tothe following drawings. The components in the drawings are notnecessarily to scale, emphasis instead being placed upon clearlyillustrating the principles of the present invention. Moreover, in thedrawings, like reference numerals designate corresponding partsthroughout the several views.

FIG. 1 is a schematic illustration of a webcam application on a computersystem.

FIG. 2 is a schematic illustration of webcam applications on twocommunicating computer systems.

FIG. 3 is a schematic illustration of the webcam application on acomputer system including the processing device and connections toperipheral devices.

FIGS. 4 a and 4 b are diagrams illustrating an effect generated based onfacial detection; FIG. 4 a shows a diagram before an effect isgenerated; and FIG. 4 b shows a user-created object associated with adetected face object.

FIG. 5 is a flow chart illustrating one embodiment of the method forgenerating effects in a webcam application.

FIGS. 6 a, 6 b, 6 c, and 6 d are diagrams illustrating certain aspectsof the embodiment of the method illustrated in FIG. 5.

FIG. 7 is a schematic illustration of the logic and hardware connectionsfor an embodiment of the system of generating effects for a webcamapplication having a display as an output.

FIG. 8 is a schematic illustration of the logic and hardware connectionsfor an embodiment of the system of generating effects for a webcamapplication having a communication as an output.

FIGS. 9 a and b are diagrams illustrating certain effects generated bythe method or system.

FIG. 10 is a flow chart illustrating another embodiment of the methodfor generating effects in a webcam application.

FIGS. 11 a, 11 b, 11 c, and 11 d are diagrams illustrating certainaspects of the method illustrated in FIG. 10.

FIG. 12 is a flow chart illustrating yet another embodiment of themethod for generating effects in a webcam application.

FIGS. 13 a and b are diagrams illustrating deforming effects possible inaddition to effects described in FIG. 12.

FIGS. 14 a and b are diagrams illustrating other effects possible inaddition to effects described in FIG. 12.

FIG. 15 is a schematic illustration of the logic and hardwareconnections for an embodiment of the system of generating effects for awebcam application having a display as an output.

FIG. 16 is a schematic illustration of the logic and hardwareconnections for an embodiment of the system of generating effects for awebcam application having a communication as an output.

FIG. 17 is a flow chart illustrating yet another embodiment of themethod for generating effects in a webcam application including trackingthe second object.

FIG. 18 is a schematic illustration of the logic and hardwareconnections for an embodiment of the system of generating effects for awebcam application including a tracking module and having a display asan output.

FIG. 19 is a schematic illustration of the logic and hardwareconnections for an embodiment of the system of generating effects for awebcam application including a tracking module and having acommunication as an output.

FIGS. 20 a, 20 b and 20 c are diagrams illustrating certain aspects ofthe method for generating effects for a webcam application includingtracking, removal and reinstatement features.

FIGS. 21 a, 21 b and 21 c are diagrams illustrating certain aspects ofthe method for generating effects for a webcam application includingdetecting an overlapping region and modifying an object in response.

DETAILED DESCRIPTION OF EMBODIMENTS

FIG. 1 illustrates a typical computer system 102 for using a webcamapplication and a user 104. The computer system 102 may include a webcam112 and a monitor 108 coupled to the processing device 106. The computersystem 102 may also include a keyboard 110 coupled to the processingdevice 106. Additionally, a mouse, although not pictured, may be coupledto the processing device 106. The monitor 108 may display a video image114. When the computer system 102 is operating, a user image 116, whichis part of a video image 114, may be captured using the webcam 112 anddisplayed on the monitor 108. The user image 116 in the video image 114captured by the webcam 112 may be displayed in real time on the monitor108. Also, an audio system 120 may be coupled to the processing device106. The audio system 120 includes a speaker 121 and a microphone 122.

FIG. 2 illustrates two users 204 a, 204 b communicating by personalcomputers 202 a, 202 b over a communication system 240. The computersystem 202 a includes a webcam 212 a and a monitor 208 a coupled to theprocessing device 206 a. The computer system 202 a may also include akeyboard 210 a and a mouse (not pictured) coupled to the processingdevice 206 a. The monitor 208 a can display a video image 214 b. Thecomputer system 202 b includes a webcam 212 b coupled to a processingdevice 206 b and a monitor 208 b also coupled to the processing device206 b. The computer system 202 b may also include a keyboard 210 b and amouse (not pictured) coupled to the processing device 206 b. The monitor208 b can display a video image 214 a. Computer system 202 a andcomputer system 202 b are both coupled to a communication system 240.

The communication system 240 may be one of various types ofcommunication systems including, for instance, the following: Internet,Intranet, Local Area Networks (LAN), Wide Area Networks (WAN) or aninterconnected combination of these network types. In addition, theconnectivity within the network 10 may be, for example, remote modem,Ethernet (IEEE 802.3), Token Ring (IEEE 802.5), Fiber DistributedDatalink Interface (FDDI), Asynchronous Transfer Mode (ATM), or anyother communication protocol.

The computing systems linked to the communication system 240 are notlimited to personal computers. The computer systems 202 a, 202 b may bea server, portable, hand-held, set-top box, personal digital assistant(PDA), a terminal, or any other desired type or configuration that has acamera and a displaying device such as a monitor or screen, etc.Depending on their functionality, the connected computer systems mayvary widely in processing power, internal memory, and other performanceaspects. Communications within the network and to or from the computersystems 202 a, 202 b connected to the communication system 240 may beeither wired or wireless. Wireless communication is especiallyadvantageous for portable or hand-held devices. The communication system240 may include, at least in part, the world-wide public Internet whichgenerally connects a plurality of users in accordance with aclient-server model in accordance with the transmission control protocolInternet protocol (TCP/IP) specification.

When the computer system 202 a is operating, a user image 216 a of auser 204 a may be captured using the webcam 212 a and sent to the secondcomputer system 202 b via a communication system 240. The secondcomputer system 202 b can then display the communicated user image 216 ain the video image 214 a on the monitor 208 b of the second computersystem 202 b. The user image 216 a captured by the webcam 212 a may bedisplayed in real time in the video image 214 b on the monitor 208 b ofthe second computer system 202 b.

Likewise, a second user 204 b can capture a second user image 216 busing a second webcam 212 b on a second computer system 202 b andcommunicate that second user image 216 b over the communication system240 to the first user 204 a. The second user image 216 b in the videoimage 214 a is displayed on the first monitor 208 a of the firstcomputer system 202 a. The second user image 216 b in the video image214 a captured by the webcam 212 a may be displayed in real time on thesecond monitor 208 b of the second computer system 202 b. In this way, afirst user 204 a and a second user 204 b can communicate using a webcamapplication.

FIG. 3 illustrates the connections between some peripheral devices andcertain logic blocks in the processing device 306. FIG. 3 shows amonitor 308 displaying a video image 314 that is connected to theprocessing device 306 and specifically connected to a display interface331 within the processing device 306. A database 318 may also beconnected to a database interface 332 within the processing device 306.The database 318 may contain a preexisting bank of objects, and theobjects may comprise image, text, video and audio files. Users canselect a user-created object from the database 318. In otherembodiments, the user-created object might also be generated by auser-drawing device (not pictured) or using a mouse (not pictured) forcreating the user-created object. The user-created object may be acopied object. In other words, the user-created object may be a regionof the video image that is copied and designated as an object.

Also shown is a keyboard 310 connected to a keyboard interface 333within the processing device 306. A mouse (not pictured) could also beconnected to the processing device 306 and would be connected to a mouseinterface (not pictured) within the processing device 306. FIG. 3further shows a webcam 312 connected to a video interface 333 within theprocessing device 306. An audio system 320 may optionally be connectedto an optional audio interface 335 within the processing device 306. Theaudio system 320 depicted in FIG. 3 shows a speaker 321 and a microphone322 for respectively outputting and inputting sound. The communicationsystem interface 336 within the processing device 306 is connected to acommunication system 340.

The interfaces (331, 332, 333, 334, 335 and 336) within the processingdevice 306 are connected to a memory 350 and a processor 370. In thememory 350 are stored audio & video interface logic 351, audio logic352, video logic 353, and a webcam application program 360. Theprocessor 370 executes the webcam application program 360 stored inmemory 350 using data acquired from keyboard 310, webcam 312, microphone322 within in the audio system 320, communication system 340, a mouse(not pictured), other data within memory or from other additionalperipheral devices.

Some webcam applications utilize facial detection for determining aportion of the video image that may correspond to a face and thenassociate an object with the detected face. For example, in FIG. 4 a, amonitor 408 a similar to the monitor 108 in FIG. 1 from a computersystem 102 may be illustrated. A video image 414 a including a userimage 416 a may be captured by webcam and displayed on a monitor 408 a.A facial detection technique may be used to detect the portion of theuser image 416 a that is the face 454 a. In FIG. 4 b, an object, amustache 452, is associated with the detected face 454 b and displayedon the face 454 b of the user image 416 b in the video image 414 b onthe monitor 408 b. Various facial detection techniques exist. Forexample, a facial detection algorithm might implement the face-detectiontask as a binary pattern-classification task. That is, the content of agiven part of an image may be categorized into features such as twoeyes, a nose and mouth, and then a classifier trained on example facesmay decide whether that particular region of the image is a face.

The flow chart of FIG. 5 shows the architecture, functionality, andoperation of a possible implementation of one embodiment of the method500 as software for generating effects for a webcam application. In thisregard, each block represents a module, segment, or portion of code,which comprises one or more executable instructions for implementing thespecified logical function(s). It should also be noted that in somealternative implementations, the functions noted in the blocks may occurout of the order illustrated in FIG. 5. For example, two blocks shown insuccession in FIG. 5 may in fact be executed substantially concurrentlyor the blocks may sometimes be executed in the reverse order, dependingupon the functionality involved. This method 500 in FIG. 5 may also beadapted to generate effects for another application using a differenttype of camera and is not limited to a webcam.

In block 561 of FIG. 5, a first object may be identified in a videoimage. The first object may be identified using various techniques suchas facial detection (if the object is a face), motion detection (if theobject moves), frame comparison analysis, edge detection analysis, orone of a variety of other object identification techniques. For example,in one technique, the difference between a frame and a next frame may befound. If the difference is less than a threshold, the video may beassumed to be static. Otherwise, the video may be assumed to be dynamic.In some embodiments, the first object may be a background region, andthis background region may be static. The video image may be analyzedand divided into a background region and a dynamic region. Thebackground region can be a static region which includes the same orsimilar frames over a predetermined period of time. Or, the backgroundregion may be captured first as a reference image, and then thisreference image may be compared to the video image that includes thereference image to determine a foreground region. Also, the backgroundregion may be identified by one of a variety of other identificationtechniques instead.

Some other examples of various techniques for object detection orbackground detection are described in the following patents which arehereby incorporated herein by reference in their entirety: U.S. Pat. No.5,748,775 issued to Tsuchikawa et al. and entitled “Method and Apparatusfor Moving Object Extraction Based on Background Subtraction”; U.S. Pat.No. 4,075,604 issued to Mario Marco Degasperi and entitled “Method andApparatus for Real Time Image Recognition”; U.S. Pat. No. 6,711,279 B1issued to Hamza et al. and entitled “Object Detection”; U.S. Pat. No.6,088,468 issued to Ito et al. and entitled “Method and Apparatus forSensing Object Located Within Visual Field of Imaging Device”; and U.S.Pat. No. 5,721,692 issued to Nagaya et al. and entitled “Moving ObjectDetection Apparatus.”

In block 562, a second object in the video image may be identified. Likethe first object, the second object may be identified by facialdetection (if the object is a face), motion detection (if the objectmoves), frame comparison analysis, edge detection analysis, or one of avariety of other object identification techniques.

In block 563, a first user-created object may be added to the videoimage to create an altered video image. The first user-created objectmight be drawn by the user or selected by a program, a user, acorrespondent, etc. from a preexisting bank of objects in a database.The first user-created object may also be generated by a program orautomatically by the webcam application. Also, the first user-createdobject may be selected from the preexisting bank of objects and thenmodified by a user, program, correspondent, etc. Further, the firstuser-created object might be text.

The first user-created object may have various characteristics. One suchcharacteristic may be a degree of deformability. In other words, thefirst user-created object could be susceptible to a high level of changein its shape or a low level of change in its shape depending upon itsdegree of deformability. Alternatively, the deformability could be of acompression-type or a bending-type. Another characteristic of theuser-created object might be temporal morphing. In other words, thefirst user-created object may change over time. For example, the firstuser-created object might be an image of a cut flower, and if displayedfor a certain amount of time, the cut flower might wilt. Anothercharacteristic of the first user-created object could be responsivenessto a stimulus.

Also described in FIG. 5 is block 564, in which a second user-createdobject may be added to the altered video image to further alter thealtered video image. The second user-created object may also have thecharacteristics described above with respect to the first user-createdobject. In addition, like the first user-created object, the seconduser-created object might be drawn by the user or selected by a program,a user, a correspondent, etc. from a preexisting bank of objects in adatabase. The second user-created object might also be generated by aprogram or automatically by the webcam application. Also, the seconduser-created object could be selected from the preexisting bank ofobjects in a database and then modified by a user, program,correspondent, etc. Further, the second user-created object might betext.

In block 565 of FIG. 5, the second user-created object may be associatedwith the second object. In certain implementations of the method, theassociation might occur automatically or could occur according to thespecification of a user, or by a hybrid of the two. The associationbetween the user-created object and the first object may be determinedby relative locations of the frames of both objects.

In block 566, a movement of the second object may be identified.Numerous motion detection techniques exist for detecting (i.e.identifying) movement in a video image and may be utilized to implementcertain features. For example, motion might be detected by comparing acurrent image with a reference image and counting the number ofdifferent pixels. Since images may naturally differ due to factors suchas varying lighting, camera flicker, and CCD dark currents, the motiondetection might include pre-processing to reduce the number of falsepositives (detecting motion when there is none).

Block 566 may also include defining a second object motion vector basedon based on the movement of the second object. This vector may includeinformation such as the direction that the second object is moving andthe velocity of the second object. Direction and velocity may bedetermined by a frame comparison analysis by comparing the location ofthe second object in different frames. An additional block that may beincluded in the method may be modifying the second user-created objectdepending on the second object motion vector described above.

Alternatively, motion may be detected based on the detection of edges invideo images (i.e. abrupt transitions in color or brightness thatdelineate one region from another. Edge detection processes and storestransitions instead of a large number of pixels, and it may takeadvantage of the high degree of correlation between pixels in a videoimage. In other words, large regions of pixels may tend to share similarvalues. An example of an edge detection system may be disclosed in U.S.Pat. No. 4,879,716 issued to Aschwanden et al. Still another possiblemotion detection technique may be subtracting the value of each pixel ofan incoming frame from the corresponding pixel in a reference frame, andaccumulating the resulting difference. Motion would be indicated whenthe accumulated difference exceeds some predetermined amount. Onereference describing various techniques may be U.S. Pat. No. 6,493,041issued to Hanko et al. Motion detection techniques other than thosedescribed here may also be suitable for use.

A motion vector may be used to describe the movement of the secondobject. In other words, the movement of the second object can have adirection and a velocity. The direction and velocity may be representedas a vector that describes the motion.

Another block included in the method 500 illustrated in FIG. 5 is block567. In block 567, the second user-created object may be moved inaccordance with the association of the second user-created object withthe second object. As discussed above, the movement of the second objectin the video image may be determined through motion detectiontechniques, and once the movement has been identified, the seconduser-created object may be moved in the video image in accordance withthe identified movement of the second object in the video image by auser, etc. In another block, the first user-created object may be movedaccording to the motion vector of the first object.

Also, in the method 500 described, the first object may be a staticobject, and the first user-created object is manually movable by a user.Also, the movement of the second user-created object in association withthe second object may be independent of the movement of the firstuser-created object.

Additionally, the method 500 may further comprise a block in which adifference of the first object and the second object is found using atechnique such as frame comparison analysis, motion detection, objectidentification, edge detection analysis, or one of a variety oftechniques for determining a difference between objects in a videoimage. Also, this difference finding may be included in one of blocks561, 562 described above.

Further, the method 500 may include modifying the first user-createdobject depending on a user's input. Likewise, the method 500 may includemodifying the second user-created object depending on a user's input.The modification may include deleting, moving, scaling, distorting,bending, compressing, stretching, shading, cropping, changing color,changing texture, or one of a variety of other modifications.

In some embodiments, a collision between two objects may be detected.For example, a collision between the first user-created object and thesecond user-created object may be detected. In addition or instead, acollision between the first user-created object and the first object maybe detected. Or, a collision between the second user-created object andthe second object may be detected. A collision between the firstuser-created object and the second object could be detected as well.

The collision may be defined in one of a variety of ways. One way mightbe to define a collision as occurring when the first user-created objectand the second user-created object are at least contiguous, at leastshare one pixel, or come within a certain distance of each other.

In response to detecting a collision, certain effects may be generated.For example, the effect could be a sound generated in response to thedetection of a collision. As another example, as an effect, acharacteristic of the first user-created object and/or the seconduser-created object could change in response to a collision detection.The first user-created object and/or the second user-created object maybe deformed in response to a collision detection.

In some embodiments, an overlap between the first user-created objectand the second object may be detected. Alternatively, in otherembodiments, an overlap between the first user-created object and thesecond user-created object may be detected. In addition or instead, anoverlap between the first user-created object and the first object maybe detected. Also in addition or instead, an overlap between the seconduser-created object and the second object may be detected.

The overlap may be defined in one of a variety of ways. One way might beto define an overlapping region as occurring where the firstuser-created object and the second object share the same portion orregion of the video image. Or, the overlapping region may be where thefirst user-created object and the second object at least share severalpixels. Or, the overlapping region may be where the first user-createdobject may be placed in front of at least a portion of the secondobject. In some embodiments, the overlapping region may occur whereother objects overlap instead of the first user-created object and thesecond object.

In response to detecting an overlap between the first user-createdobject and the second object, the first user-created object may bemodified. This modification may be deformation, deletion, movement,scaling, distortion, bending, compression, stretching, shading,cropping, color change, texture change, or one of a variety of othermodifications. FIGS. 21 a, 21 b and 21 c illustrate one example ofmodification in response to detecting an overlap between a firstuser-created object 2119 and a second object 2191. In FIG. 21 a, a firstobject 2118 a, which may be a background region, has been identified. Afirst user-created object 2119 a, which may be a star image, may havebeen added to the video image 2114 a and associated with the firstobject 2118 a. Additionally, a second object 2191 a, which may be a handimage, may have been identified and a second user-created object 2192 a,which may be a soccer ball image, may have been associated with thesecond object 2191 a. In FIG. 21 b, the second object 2191 b may havemoved in the video image 2114 b such that the hand image overlaps 2194 ba portion of the first user-created object 2119 b. FIG. 21 c illustratesa modification of the first user-created object 2119 c that may occur inresponse to the second object 2191 b overlapping the first user-createdobject 2119 c. The modification depicted may be an increase in the scaleof the first user-created object 2119 c. In other words, the size of thestar image may have increased in response to the overlapping of thesecond object 2191 c and the first user-created object 2119 c. In otherembodiments, the modification may have been deformation, deletion,movement, distortion, bending, compression, stretching, shading,cropping, color change, texture change, or one of a variety of othermodifications. Also, a different combination of objects may have formedthe overlapped region, and one or more of those objects may be modifiedin response to a detection of the overlap.

In some embodiments, the method may also include tracking the secondobject. The tracking feature may include a determination of whether thesecond object was in the video image. Or, the tracking might determinewhere the second object is within the video image. In addition, thesecond user-created object may be removed responsive to a disappearanceof the tracked second object from the video image. Also, the seconduser-created object may be reinstated responsive to a reappearance ofthe tracked second object in the video image. The reinstatement mayinclude placing the removed second user-created object in the sameassociation with respect to the second object as before the removal ofthe second user-created object. In some embodiments, this may meanputting the second user-created object back in the same location as thesecond object.

FIGS. 20 a, 20 b, and 20 c illustrate one nonlimiting example of theembodiment of the tracking, removal and reinstatement features describedabove. FIG. 20 a shows a monitor 2008 a displaying a video image 2014 a.The video image 2014 a includes a first object 2018 a, which is abackground region, and a first user-created object 2019 a, which is astar image, associated with the background region 2018 a. Also includedin the image is a second object, 2091 a, which is a hand image, and asecond user-created object 2092 a, which is a soccer ball image. Asshown in FIG. 20 a, the second object 2091 a may be tracked. In otherwords, whether the second object 2091 a is in the video image 2014 a maybe determined. In some embodiments, whether at least portion of thesecond object 2091 a is in the video image 2014 a and/or the amount ofthe portion of the second object 2091 a that is present in the videoimage 2014 a may be determined as part of the tracking feature.

FIG. 20 b illustrates the second object having moved out of the videoimage 2014 b. Thus, the second object 2091 a, which is the hand image,displayed in the video image 2014 a in FIG. 20 a is not displayed in thevideo image 2014 b in FIG. 20 b. As shown in FIG. 20 b, the secondobject has disappeared from the video image 2014 b and then, the seconduser-created object 2092 a, which was displayed in video image 2014 a inFIG. 20 a, may be removed from the video image 2014 b in FIG. 20 b.Also, as shown in FIG. 20 c, If the second object 2091 c reappears inthe video image 2014 c, then the second user-created object 2092 c maybe reinstated in the video image 2092 c. In this embodiment, the firstobject 2018 and the first user-created object 2019 may be independent ofthe second object 2091 and the second user-created object 2092.

In some of these embodiments, if only a portion of the second objectdisappears, then only corresponding portion of the second user-createdobject will be removed. Similarly, in some of these embodiments, if onlya portion of the second object reappears, then only a correspondingportion of the second user-created object will reappear.

FIGS. 6 a,b,c,d illustrates a nonlimiting example of the embodimentdescribed in FIG. 5. As an example, FIG. 6 a may show a monitor 608 adisplaying a video image 614 a including a user image 616 a. Accordingto block 561 of FIG. 5, a first object 618 a may be identified in avideo image 614 a, and in this illustration, the first object 618 a maybe a background region. According to block 562 of the method illustratedin FIG. 5, a second object 691 a may be identified in the video image614 a. In this example, the second object 691 a identified may be a handimage that is part of the user image 616 a. The hand image may bedetected by motion detection or one of a variety of other objectdetection techniques.

FIG. 6 b illustrates the block 563 in which a first user-created object619 b may be added to the video image 614 a to create an altered videoimage 614 b. In this example, the first user-created object 619 b may bea star image. In accordance with the adding block 564 of FIG. 5, FIG. 6b illustrates a soccer ball as a second user-created object 692 b addedto the video image 614 a. A further altered video image 614 b resultsfrom the addition and is shown as a user image 616 b with a backgroundregion plus a soccer ball and a star. Then, according to block 565 ofFIG. 5, the second user-created object 692 b, the soccer ball, may thenbe associated with the second object 691 b, the hand image.

Per the identifying movement block 566 of FIG. 5 and as shown in FIG. 6c, the movement 664 c of the second object 691 c in the video image 614c may be identified. In this nonlimiting example, the first object 618c, which may be the background region, remains static, and the firstuser created object 619 c may remain static as well. Then, as indicatedin block 567 of FIG. 5 and as shown in FIG. 6 d, the second user-createdobject 692 d may be moved 665 d in accordance with the association ofthe second user-created object 692 d and the second object 691 d. Inthis example, the second user-created object 692 d, the soccer ball, maybe moved 665 d in the same direction as the movement 664 d of the secondobject 691 d, the hand. However, the direction, speed, etc. of themovement of the second user-created object need not be identical to thatof the second object. Please note that, when the user or user's handmoves in the video image, the star added to the static background regionmay remain at the same location and may be independent from the movementof the second object (hand or user) or the second user-created object(soccer ball). Also, the star may be manually movable by the user andrepositioned in a different location on the background region.

FIG. 7 illustrates a logic and hardware diagram of one embodiment of thesystem for generating effects for a webcam application. This diagramshows the webcam application 760 being executed by the processor 770 inthe processing device 706 and shows certain inputs to and outputs fromthe processor 770. Although this figure does not illustrate the computersystem 702 in communication with any other computer system, such aconfiguration may be possible.

In this illustration of the system, a webcam 712 may send a video image780 to the processing device 706, and the video image 780 may bereceived by the video interface 733. The video interface 733 may thensend the video image 780 to an identification module 761 correspondingto the webcam application 760 on the processor 770. This identificationmodule 761 may contain logic for identifying a first object 791 and asecond object 718 in a video image 780. Once the first object 791 andsecond object 718 are identified, the video image 780 containing thefirst object 791 and second object 718 may be sent to an addition module763. The addition module 763 may add a first user-created object 792 tothe video image 780 and a second user-created object 719 to the videoimage 780 containing the identified first object 791 and second object718 to create an altered video image 781. A user may input informationregarding the first user-created object 792 into a keyboard 710, whichmay be connected to the processing device 706. Likewise, a user mayinput information regarding the second user-created object 719 into akeyboard 710, which may be connected to the processing device 706. Thesignal from the keyboard 710 may be fed into the keyboard interface 732of the processing device 706. Also, though not pictured, a mouse may beconnected to the processing device 706 and may receive user inputregarding the second user-created object 719. The signal from the mousemay be fed into a mouse interface (not pictured) of the processingdevice 706. Thus, the user-created objects 792, 719 may be sent to thekeyboard interface 732 via the connection, and the keyboard interface732 may send the user-created objects 792, 719 to the addition module763. The addition module 763 may include logic for adding the firstuser-created object 792 to the video image 780 to create an alteredvideo image 781. Also, the addition module 763 may include logic foradding the second user-created object 719 to the altered video image781, which may further alter the altered video image 781. The alteredvideo image 781 may then be sent to an association module 765. Theassociation module 765 may include logic for associating the seconduser-created object 719 with the second object 718. After the seconduser-created object 719 may be associated with the second object 718,the altered video image 782 may be sent to a motion detection module766. The motion detection module 766 may contain logic for identifying amovement of the second object 718 in the altered video image 782. Thismotion detection may be accomplished using one of a variety of motiondetection techniques as described above. Then, the altered video image783 may be sent to a movement module 767. The movement module 767 maycontain logic for moving the second user-created object 719 inaccordance with the association of the second user-created object 719with the second object 718. Thus, the second user-created object 719 maybe moved in the altered video image 784, and this altered video image784 may be sent to a display interface 731. The display interface 731may then send this altered video image 784 to the monitor 708, which maydisplay the altered video image 784.

In some embodiments, if no motion of the first object 791 was detectedin the motion detection module 766, the altered video image 784 may besent to the movement module 767, which may not change the movement ofthe second user-created object 719. The altered video image 784 showingno movement of the second object 718 or the second user-created object719 may be sent to the display interface 731, which may send thataltered video image 784 to the monitor 708. The monitor 708 may thendisplay the altered video image 784 showing no movement of the secondobject 718 or the second user-created object 719.

FIG. 8 also illustrates a logic and hardware diagram of one embodimentof a system for generating effects for a webcam application. Thisdiagram shows the webcam application 860 being executed by the processor870 in the processing device 806 and shows certain inputs to and outputsfrom the processor 870. This figure illustrates the computer system 802communicating via the communication system 840.

FIG. 8 is similar to FIG. 7 except that the output may be sent to acommunication system 840 instead of a monitor 708. Specifically, thealtered video image 884 may be sent to the communication systeminterface 810, which sends the altered video image 884 to thecommunication system 840. Although, a monitor is not illustrated, onemay be attached as well as a peripheral device. The altered video image884 sent to the communication system interface 810, and thecommunication system 840 could also be sent to a connected monitor aswell.

In addition to the method 500 described in FIG. 5, another block couldinclude changing certain characteristics of the second user-createdobject according to changes in the associated second object. Forexample, if the scale of the second object changes, the scale of thesecond user-created object may be changed. This possible additionalfeature of the method 500 may be illustrated in FIGS. 9 a and 9 b. Amonitor 908 a of a computer system displays the altered video image 914a including a first object 918 a, which may be the background region; asecond object 991 a, which may be a hand image; a first user-createdobject 919 a, which may be a star image, and a second user-createdobject 992 a, which may be a soccer ball image. When the user moves herhand toward the webcam, the scale of the hand image may be increased inthe video image 914 b. In other words, the size of the second object 991b may appear larger in the video image 914 b. Because thischaracteristic, the scale of the second object 991 b, changed, the samecharacteristic of the second user-created object 992 b, the soccer ballimage, may be changed as well. In FIG. 9 b, the size of the hand imagemay be increased, and the size of the soccer ball image correspondinglymay be increased. This change may give the appearance that, like thehand that has moved closer to the webcam, the soccer ball associatedwith the hand may have also moved closer to the webcam. Please notethat, the scale of the star on the background may be independent fromthe scaling of the second user-created object 992 b, the soccer ballimage.

Though not illustrated, a similar example of changing a characteristicof the second user-created object according to a change in the secondobject may be changing the shading of a second user-created objectaccording to the shading of the second object. There may be a shadingfactor that corresponds to the second object and a shading factor thatcorresponds to the second user-created object. Whenever the shadingfactor of the second object changes, the shading factor of the seconduser-created object may also be changed. This shading change of thesecond user-created object may be independent of any shading or shadingchanges of the first object and first user-created object.

In still another example not illustrated, if the second user-createdobject is text, the text may change depending on a characteristic of thesecond object. An example of this might be text associated with a handimage, and as the hand moves to a different location in the video image,the word in the text could correspondingly change. For instance, thehand image could be located at a top portion of the video image, and thehand image could be associated with text that says “HIGH.” If the handregion moves down to a bottom portion of the video image, the text couldbe changed to say “LOW.” Again, this change in the second user-createdobject may be independent of the first user-created object and the firstobject.

The flow chart of FIG. 10 shows the architecture, functionality, andoperation of a possible implementation of one embodiment of the method1000 as software for generating effects for a webcam application. Inthis regard, each block represents a module, segment, or portion ofcode, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat in some alternative implementations, the functions noted in theblocks may occur out of the order illustrated in FIG. 10. For example,two blocks shown in succession in FIG. 10 may in fact be executedsubstantially concurrently or the blocks may sometimes be executed inthe reverse order, depending upon the functionality involved. Method1000 may also be adapted to generate effects for another applicationusing a different type of camera and is not limited to a webcam.

The method 1000 may include block 1061. In block 1061, a first objectand a second object may be identified in a video image. The objectscould be identified using various techniques such as facial detection(if the object is a face), motion detection (if the object moves) oranother object identification technique. For example, in one technique,the difference between a frame and a next frame may be found. If thedifference is less than a threshold, the video may be assumed to bestatic. Otherwise, the video may be assumed to be dynamic. In someembodiments, the first object may be a background region, and thisbackground region may be static. The video image may be analyzed anddivided into a background region and a dynamic region. The backgroundregion can be a static region which includes the same or similar framesover a predetermined period of time. Or, the background region may becaptured first as a reference image, and then this reference image maybe compared to the video image that includes the reference image todetermine a foreground region. Also, the background region may beidentified by one of a variety of other identification techniquesinstead.

Some other examples of various techniques for object detection orbackground detection are described in the following patents which arehereby incorporated herein by reference in their entirety: U.S. Pat. No.5,748,775 issued to Tsuchikawa et al. and entitled “Method and Apparatusfor Moving Object Extraction Based on Background Subtraction”; U.S. Pat.No. 4,075,604 issued to Mario Marco Degasperi and entitled “Method andApparatus for Real Time Image Recognition”; U.S. Pat. No. 6,711,279 B1issued to Hamza et al. and entitled “Object Detection”; U.S. Pat. No.6,088,468 issued to Ito et al. and entitled “Method and Apparatus forSensing Object Located Within Visual Field of Imaging Device”; and U.S.Pat. No. 5,721,692 issued to Nagaya et al. and entitled “Moving ObjectDetection Apparatus.”

Block 1061 may also include identifying a third object, fourth object,etc. in the video image in addition to identifying the first object andthe second object in the video image. In block 1062, a firstuser-created object may be added to the first object in the video imageto create an altered video image. The first user-created object might becreated by the user or selected by a program, a user, a correspondent,etc. from a preexisting bank of objects in a database. The firstuser-created object might also be generated by a program orautomatically by the webcam application. Also, the first user-createdobject may be selected from the preexisting bank of objects and thenmodified by a user, program, correspondent, etc. before the addition.Further, the first user-created object may be text.

Further, the first user-created object could also have variouscharacteristics. One such characteristic could be a degree ofdeformability. In other words, as an effect, the first user-createdobject may be susceptible to a high level of change in its shape or alow level of change in its shape depending upon a degree ofdeformability. Alternatively, the deformability may be of acompression-type or a bending-type. Another characteristic of the firstuser-created object may be temporal morphing as an effect. In otherwords, the first user-created object may change over time. For example,the first user-created object might be an image of a cut flower, and ifdisplayed for a certain amount of time, the cut flower might wilt.Another characteristic of the first user-created object may beresponsiveness to a stimulus as an effect. A characteristic may dependon whether the first user-created object is located in the foreground orthe background. For example, a first user-created object located on thebackground may remain static, while a first user-created object in theforeground may be moved.

In addition to block 1062, block 1063 may be performed. In block 1063, asecond user-created object may be added to the second object to furtheralter the altered video image. Like the first user-created object, thesecond user-created object may have some of the characteristicsdescribed above. Also, the second user-created object might be createdby the user or selected by a program, a user, a correspondent, etc. froma preexisting bank of objects in a database. The second user-createdobject might also be generated by a program or automatically by thewebcam application. Also, the second user-created object may be selectedfrom the preexisting bank of objects and then modified by a user,program, correspondent, etc. before the addition. Further, the seconduser-created object might be text.

Another block in the method 1000 is block 1064. In block 1064, the firstuser-created object may be associated with the first object identifiedin block 1061. In some implementations of the method, the associationmight occur automatically, could occur according to the specification ofa user, or by a hybrid of the two. In block 1065, the seconduser-created object may be associated with the second object.Additionally, if third object, fourth object, and etc. were identified,the first user-created object or second user-created object may beassociated with those objects.

In block 1066, a movement of the first object in the video image may beidentified (i.e. detected). A movement of a second object in the videoimage may also be detected. The movement of the first object and thesecond object may be independent. Numerous motion detection techniquesexist for detecting movement in a video image as discussed above. Thosemotion detection techniques are also applicable in this embodiment.

Another block included in the method 1000 is block 1067. This block mayinclude moving the first user-created object in accordance with theassociation of the first user-created object with the first object. Asdiscussed above, the movement of the first object in the video imagemight be determined through motion detection techniques, and once themovement is identified, as an effect, the first user-created object maybe moved in the video image in accordance with the identified movementof the first object in the video image, etc. Also, the seconduser-created object may be moved as an effect in accordance with theassociation of whichever identified object with which it becameassociated. The movement of the second user-created object may beindependent of the movement of the first user-created object. The firstuser-created object and second user-created objects may also be manuallymovable by a user.

FIGS. 11 a,b,c,d illustrate certain aspects of the embodiment shown inFIG. 10. FIG. 11 shows a monitor 1108 a displaying a video image 1114 a.According to the identifying block, a first object 1191 a, which is ahand image, may be identified. A second object 1168 a, which may be ahead image, may be identified. After the hand image and the head imageare identified, a first user-created object 1169 b, which is a rectanglein this illustration, may be added to the video image 1114 b asillustrated in FIG. 11 b. The first user-created object 1169 b may alsobe associated with the first object 1191 b, a hand image. A seconduser-created object 1166 b, in this case a halo, may also be added tothe video image 1114 b. This illustration shows the halo associated withthe head image, which may be the second object 1168 b identified.

Then, FIG. 11 c illustrates the identification of movement of the firstobject 1191 c. The movement 1164 c of the hand image may be identified.The movement of the second object 1168 c, the head image, may also beidentified. This feature may be illustrated in FIG. 11 c as a movement1167 c of a head image being identified.

In FIG. 11 d, as an effect, the first user-created object 1169 d, therectangle, may be moved 1180 d in accordance with the association of thefirst user-created object 1169 d and the first object 1191 d, the handimage. Additionally, the second user-created object 1166 d, the halo,may be moved 1181 d in accordance with an association of the secondobject 1168 d, the head image, having a movement 1167 d, with the seconduser-created object 1166 d.

The flow chart of FIG. 12 shows the architecture, functionality, andoperation of a possible implementation of another embodiment of themethod as software for generating effects for a webcam application. Inthis regard, each block represents a module, segment, or portion ofcode, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat in some alternative implementations, the functions noted in theblocks may occur out of the order noted in FIG. 12. For example, twoblocks shown in succession in FIG. 12 may in fact be executedsubstantially concurrently or the blocks may sometimes be executed inthe reverse order, depending upon the functionality involved. Method1200 could also be adapted to generate effects for another applicationusing a different type of camera and is not limited to a webcam.

The method 1200 shown in FIG. 12 may be similar to the method 1000illustrated in FIG. 10. The method 1200 includes block 1261. In block1261 a first object may be identified in a video image. The object couldbe identified using various techniques such as facial detection (if theobject is a face), motion detection (if the object moves) or anotherobject identification technique. A second object, third object, fourthobject, etc. may also be identified in the video image in addition toidentifying the first object in the video image.

In block 1262, a user-created object may be added to the video image tocreate an altered video image. The user-created object might be createdby the user or may be selected by a program, a user, a correspondent,etc. from a preexisting bank of objects in a database. The user-createdobject might also be generated by a program or automatically by thewebcam application including a user-drawing module. Also, theuser-created object could be selected from the preexisting bank ofobjects and then modified by a user, program, correspondent, etc.Further, the user-created object might be text.

The user-created object may also have various characteristics. One suchcharacteristic could be a degree of deformability. In other words, theuser-created object may be susceptible to a high level of change in itsshape or a low level of change in its shape depending upon a degree ofdeformability. Alternatively, the deformability could be of acompression-type or a bending-type. Another characteristic of theuser-created object might be temporal morphing. In other words, theuser-created object may change over time. For example, the user-createdobject might be an image of a cut flower, and if displayed for a certainamount of time, the cut flower might wilt. Another characteristic of theuser-created object could be responsiveness to a stimulus.

In addition to block 1262, the actions in block 1263 may be performed.In block 1263, a second user-created object may be added to the alteredvideo image. Another block in the method 1200 is block 1264. In theassociating block 1264, the user-created object may be associated withthe first object identified in the identifying block 1261. In animplementation of the method 1200, the association might occurautomatically or could occur according to the specification of a user,or by a hybrid of the two. Also, in block 1265, a second user-createdobject may be associated with the first object, the first user-createdobject, or a second object. If a third object, fourth object, etc. wereidentified, the first user-created object or second user-created objectmay be associated with those objects as well or instead.

In block 1266, a movement of the first object in the video image may bedetected (i.e. identified). A movement of a second object may also bedetected if a second object was identified. Numerous motion detectiontechniques exist for detecting movement in a video image as discussedabove. Those motion detection techniques are also applicable in thisembodiment. A first motion vector may be used to describe the motion ofthe first object, and a second motion vector may be used to describe themotion of the second object. In other words, the motion of each objectcan have a direction and a velocity. The direction and velocity of eachcan be represented as a vector that describes the motion of each.

Another block included in the method 1200 is block 1267. In this block,the first user-created object may be moved in accordance with theassociation of the first user-created object with the first object. Asdiscussed above, the movement of the first object in the video imagemight be determined through motion detection techniques, and once themovement is identified, the user-created object may be moved in thevideo image in accordance with the identified movement of the firstobject in the video image. by a user, etc. Also, the second user-createdobject may be moved in accordance with the association of whicheveridentified object it may have been associated with. The movement of thesecond user-created object may be independent of the objects with whichthe second user-created object is not associated.

Yet another possible block in the method 1200 may be block 1268. Inblock 1268, a collision between two objects may be detected. Forexample, a collision between the first user-created object and thesecond user-created object may be detected. In addition or instead, acollision between the first user-created object and the first object maybe detected. Or, a collision between the second user-created object andthe second object may be detected. A collision between the firstuser-created object and the second object could be detected as well.

The collision may be defined in one of a variety of ways. One way mightbe to define a collision as occurring when the first user-created objectand the second user-created object are at least contiguous, at leastshare one pixel, overlap or come within a certain distance of eachother.

In response to detecting a collision, certain effects may be generated.For example, the effect could be a sound generated in response to thedetection of a collision. As another example, as an effect, acharacteristic of the first user-created object and/or the seconduser-created object could change in response to a collision detection.The first user-created object and/or the second user-created object maybe deformed in response to a collision detection.

FIGS. 13 a and 13 b illustrate one nonlimiting example of this feature.FIG. 13 a shows a collision 1368 a between the second user-createdobject 1366 a, the halo, and the first user-created object 1369 a, therectangle. Here, the rectangle and the halo may be shown in the videoimage 1314 a as being at least contiguous. In response to the detectedcollision, the second user-created object 1366 b, may be deformed 1369 bas an effect generated due to the collision 1368 b. Although not shownin the figure, when a collision between the first user-created object(the rectangle 1369 a of FIG. 13) and the first object (the head of thefirst object 1314 a) is detected, the first user-created object may bedeformed in response to the collision detection.

FIGS. 14 a and 14 b illustrate another nonlimiting example of effectsgeneration feature. In FIGS. 14 a and 14 b a typical computer system1402 a, 1402 b for using a webcam application is shown. The computersystem 1402 a, 1402 b may include a webcam 1412 a, 1412 b and a monitor1408 a, 1408 b coupled to the processing device 1406 a, 1406 b. Thecomputer system 1402 a, 1402 b may also include a keyboard 1410 a, 1410b coupled to the processing device 1406 a, 1406 b. Additionally, amouse, although not pictured, may be coupled to the processing device1406 a, 1406 b. The monitor 1408 a, 1408 b may display a video image1414 a, 1414 b. Also, an audio system 1420 a, 1420 b may be coupled tothe processing device 1406 a, 1406 b. The audio system 1420 a, 1420 b,may include a speaker 1421 a, 1421 b and a microphone 1422 a, 1422 b.

FIG. 14 a illustrates a first object 1491 a, a hand image, and a secondobject 1168 a, a head image. A first user-created object 1469 a, whichis a rectangle, may have been added to the video image 1414 a and may beassociated with the first object 1491 a. In this illustration the firstuser-created object 1469 a may be moving. FIG. 14 b illustrates acollision between the first user-created object 1469 b and the firstobject 1491 b. In response to detecting a collision between the firstuser-created object 1469 b and the first object 1491 b, an effect may begenerated. That effect may be a playing of a sound 1423 b. The soundcould be a crash, explosion, or alarm-type sound, for example.

Alternatively, text could be displayed in response to detecting acollision. The text might contain an onomatopoeia that describes a soundsuch as “KA-BOOM” or “SPLAT”. Also, both the text object and the soundcould be displayed as dual effects. Additionally, instead of playing asound or displaying text, a third user-created object could be displayedin response to detecting a collision. Also, in response to detecting acollision, one or more of the user-created objects could change color,become transparent or change position.

The effect generated could also depend on the relative motion vectors ofthe first object and the second object. For example, if the first objectis associated with a first user-created object and the second object isassociated with a second user-created object, the movement of the firstobject relative to the movement of the second object may affect theeffect generated in response to detecting a collision between the firstuser-created object and the second user-created object. If the firstobject and second object are moving toward each other at a high velocitywhen the first user-created object and the second user-created objectcollide, the effect generated might be of a higher intensity. If thefirst object is moving and the second object is not, and a collisionbetween the first user-created object and the second user-created objectresults, the effect generated might be of a lower intensity. The motionvectors of the first object and the second object may be compared todetermine the appropriate intensity of the effect generated. Theintensity of the effect generated may be the volume of the sound, theamount that the shapes of the user-created objects are deformed, thesize of the text displayed or a third user-created object displayed,etc.

Another effect that could be generated may be showing a shaky image ofthe first user-created object or the second user-created object. Theshaky image could represent vibrations that result due to the fictitiousimpact that results out of the collision. The shaky image would be aquick series of slightly moved images of the user-created object to givethe appearance of shaking or vibrations. Or the objects themselves couldbe displayed as shaking by showing the objects as slightly moved backand forth in a series of frames. Alternatively, the entire video imagecould be shown as a shaky image in response to detecting a collision.

The shaking of the display may have a degree or intensity as well. Theintensity could be indicated by how far the images are moved from theiroriginal positions as they move back and forth. This intensity maydepend upon the relative motion vectors of the first object and thesecond object. For example, if both the first object and the secondobjects are heading toward each other (in other words, in oppositedirections) at a high velocity, the intensity of the shaking of thefirst object, the second object, or both may be high.

FIG. 15 illustrates a logic and hardware diagram of one embodiment ofthe system for generating effects for a webcam application. This diagramshows the webcam application 1560 being executed by the processor 1570in the processing device 1506 and shows certain inputs to and outputsfrom the processor 1570. The processor 1570 outputs the altered videoimage 184 to a display interface 1531, which then sends it to a monitor1508. The monitor 1508 will display the altered video image 1584.Although not illustrated in this diagram, the computer system 1502 mayalso be in communication with another computer system.

In FIG. 15, a webcam 1512 sends a video image 1580 to the processingdevice 1506 and the video image 1580 is received by the video interfacemodule 1561. The video interface 1533 then sends the video image 1580 toan identification module 1561. This identification module 1561 containslogic for identifying a first object 1591 and a second object 1518 in avideo image 1580. Once the first object 1591 and the second object 1518have been identified, the video image 1580 containing the identifiedobjects may be sent to an addition module 1563. The addition module 1563may add a first user-created object 1592 and a second user-createdobject 1519 to the video image 1580 to create an altered video image1581. Then, the altered video image 1581 may be sent to an associationmodule 1565. The association module 1565 may associate the firstuser-created object 1592 with the first object 1591, and the associationmodule 1565 may also associate the second user-created object 1519 withthe second object 1518 in video image 1582. Then video image 1582 may besent to a motion detection module 1566. The motion detection module 1566may detect a movement of the first object 1591 and/or the second object1518. Then the video image 1583 may be sent to a collision detectionmodule 1567. In the collision detection module 1567, a collision betweenthe first object 1591 and the first user-created object 1592 may bedetected. Alternatively, a collision between the first object 1591 andthe second object 1518 may be detected. Or, a collision between thesecond object 1518 and the second user-created object 1519 may bedetected. Instead, a collision between the first user-created object1592 and the second user-created object 1519 may be detected. Or, acollision between the first user-created object 1592 and the secondobject 1518 may be detected. Also, a collision between the seconduser-created object 1519 and the first object 1591 may be detected inusing the collision detection module 1567.

Then, the video image 1583 may be sent to an effects generation module1567. The effects generation module 1567 may generate an effect inresponse to the detection of a collision by the collision detectionmodule 1567. The effect generated may be one of the effects describedabove. Then the video image 1584 may be sent to a display interface1531, which may send the video image 1584 to a monitor 1508 fordisplaying the video image 1584. Alternatively or in addition, asillustrated in FIG. 16, the video image 1684 may be sent to thecommunication system interface 1610, which may then send the video image1684 to a communication system 1640.

The flow chart of FIG. 17 shows the architecture, functionality, andoperation of a possible implementation of another embodiment of themethod as software for generating effects for a webcam application. Inthis regard, each block represents a module, segment, or portion ofcode, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat in some alternative implementations, the functions noted in theblocks may occur out of the order noted in FIG. 17. For example, twoblocks shown in succession in FIG. 17 may in fact be executedsubstantially concurrently or the blocks may sometimes be executed inthe reverse order, depending upon the functionality involved. Method1700 could also be adapted to generate effects for another applicationusing a different type of camera and is not limited to a webcam.

The method 1700 shown in FIG. 17 may be similar to the method 1000illustrated in FIG. 10. The method 1700 includes block 1761. In block1761 a first object may be identified in a video image. The object couldbe identified using various techniques such as facial detection (if theobject is a face), motion detection (if the object moves) or anotherobject identification technique. A second object, third object, fourthobject, etc. may also be identified in the video image in addition toidentifying the first object in the video image.

In block 1762, a user-created object may be added to the video image tocreate an altered video image. The user-created object might be createdby the user or may be selected by a program, a user, a correspondent,etc. from a preexisting bank of objects in a database. The user-createdobject might also be generated by a program or automatically by thewebcam application including a user-drawing module. Also, theuser-created object could be selected from the preexisting bank ofobjects and then modified by a user, program, correspondent, etc.Further, the user-created object might be text.

The user-created object may also have various characteristics. One suchcharacteristic could be a degree of deformability. In other words, theuser-created object may be susceptible to a high level of change in itsshape or a low level of change in its shape depending upon a degree ofdeformability. Alternatively, the deformability could be of acompression-type or a bending-type. Another characteristic of theuser-created object might be temporal morphing. In other words, theuser-created object may change over time. For example, the user-createdobject might be an image of a cut flower, and if displayed for a certainamount of time, the cut flower might wilt. Another characteristic of theuser-created object could be responsiveness to a stimulus.

In addition to block 1762, the actions in block 1763 may be performed.In block 1763, a second user-created object may be added to the alteredvideo image. Another block in the method 1700 is block 1764. In theassociating block 1764, the user-created object may be associated withthe first object identified in the identifying block 1761. In animplementation of the method 1700, the association might occurautomatically or could occur according to the specification of a user,or by a hybrid of the two. Also, in block 1765, a second user-createdobject may be associated with the first object, the first user-createdobject, or a second object. If a third object, fourth object, etc. wereidentified, the first user-created object or second user-created objectmay be associated with those objects as well or instead.

In block 1766, a movement of the first object in the video image may bedetected (i.e. identified). A movement of a second object may also bedetected if a second object was identified. Numerous motion detectiontechniques exist for detecting movement in a video image as discussedabove. Those motion detection techniques are also applicable in thisembodiment. A first motion vector may be used to describe the motion ofthe first object, and a second motion vector may be used to describe themotion of the second object. In other words, the motion of each objectcan have a direction and a velocity. The direction and velocity of eachcan be represented as a vector that describes the motion of each.

Another block included in the method 1700 is block 1767. In this block,the first user-created object may be moved in accordance with theassociation of the first user-created object with the first object. Asdiscussed above, the movement of the first object in the video imagemight be determined through motion detection techniques, and once themovement is identified, the user-created object may be moved in thevideo image in accordance with the identified movement of the firstobject in the video image. by a user, etc. Also, the second user-createdobject may be moved in accordance with the association of whicheveridentified object it may have been associated with. The movement of thesecond user-created object may be independent of the objects with whichthe second user-created object is not associated.

Yet another possible block in the method 1700 may be block 1769. Inblock 1769, the second object may be tracked. The tracking feature mayinclude a determination of whether the second object was in the videoimage. Or, the tracking might determine where the second object iswithin the video image. In addition, in block 1770, the seconduser-created object may be removed responsive to a disappearance of thetracked second object from the video image. Also, in block 1771, thesecond user-created object may be reinstated responsive to areappearance of the tracked second object in the video image. In some ofthese embodiments, if only a portion of the second object disappears,then only corresponding portion of the second user-created object willbe removed. Similarly, in some of these embodiments, if only a portionof the second object reappears, then only a corresponding portion of thesecond user-created object will reappear.

FIG. 18 illustrates a logic and hardware diagram of one embodiment ofthe system for generating effects for a webcam application. This diagramshows the webcam application 1860 being executed by the processor 1870in the processing device 1806 and shows certain inputs to and outputsfrom the processor 1870. The processor 1870 outputs the altered videoimage 1884 to a display interface 1831, which then sends it to a monitor1808. The monitor 1808 will display the altered video image 1884.Although not illustrated in this diagram, the computer system 1802 mayalso be in communication with another computer system.

In FIG. 18, a webcam 1812 sends a video image 1880 to the processingdevice 1806 and the video image 1880 is received by the video interfacemodule 1861. The video interface 1833 then sends the video image 1880 toan identification module 1861. This identification module 1861 containslogic for identifying a first object 1891 and a second object 1818 in avideo image 1880. Once the first object 1891 and the second object 1818have been identified, the video image 1880 containing the identifiedobjects may be sent to an addition module 1863. The addition module 1863may add a first user-created object 1892 and a second user-createdobject 1819 to the video image 1880 to create an altered video image1881. Then, the altered video image 1881 may be sent to an associationmodule 1865. The association module 1865 may associate the firstuser-created object 1892 with the first object 1891, and the associationmodule 1865 may also associate the second user-created object 1819 withthe second object 1818 in video image 1882. Then video image 1882 may besent to a motion detection module 1866. The motion detection module 1866may detect a movement of the first object 1891 and/or the second object1818. Then, the altered video image 1883 may be sent to a movementmodule 1867. The movement module 1867 may contain logic for moving thesecond user-created object 1819 in accordance with the association ofthe second user-created object 1819 with the second object 1818. Thus,the second user-created object 1819 may be moved in the altered videoimage 1884. Next, this altered video image 1884 may be sent to atracking module 1869.

In the tracking module 1869, the second object 1818 may be tracked.Also, the second user-created object 1819 may be removed in response toa disappearance of the tracked second object 1818 from the video image.Then, the second user-created object 1819 may be reinstated in responseto the reappearance in the video image of the tracked second object1818. Later, the video image 1884 may be sent to a display interface1831, which may send the video image 1884 to a monitor 1808 fordisplaying the video image 1884. Alternatively or in addition, asillustrated in FIG. 19, the video image 1984 may be sent to thecommunication system interface 1910, which may then send the video image1984 to a communication system 1940.

FIGS. 20 a, 20 b, and 20 c may be discussed with respect to theembodiment described in FIG. 17. FIG. 20 a shows a monitor 2008 adisplaying a video image 2014 a. The video image 2014 a includes a firstobject 2018 a, which is a background region, and a first user-createdobject 2019 a, which is a star image, associated with the backgroundregion 2018 a. Also included in the image is a second object, 2091 a,which is a hand image, and a second user-created object 2092 a, which isa soccer ball image. According to block 1769, the second object 2091 amay be tracked. In other words, whether the second object 2091 a is inthe video image 2014 a may be determined. In some embodiments, whetherat least portion of the second object 2091 a is in the video image 2014a and/or the amount of the portion of the second object 2091 a that ispresent in the video image 2014 a may be determined as part of block1769.

FIG. 20 b illustrates the second object having moved out of the videoimage 2014 b. Thus, the second object 2091 a, which is the hand image,displayed in the video image 2014 a in FIG. 20 a is not displayed in thevideo image 2014 b in FIG. 20 b. In accordance with block 1770, thesecond object has disappeared from the video image 2014 b and then, thesecond user-created object 2092 a, which was displayed in video image2014 a in FIG. 20 a, may be removed from the video image 2014 b in FIG.20 b. Also, in accordance with block 1771, if the second object 2091 creappears in the video image 2014 c as illustrated in FIG. 20 c, thenthe second user-created object 2092 c may be reinstated in the videoimage 2092 c. In this embodiment, the first object 2018 and the firstuser-created object 2019 may be independent of the second object 2091and the second user-created object 2092.

In some embodiments, both the first object may be tracked and the secondobject may be tracked. Also, the first user-created object may beremoved if the first object disappears from the video image. Inaddition, the second user-created object may be removed if the secondobject disappears from the video image. In other words, if the userdisappears from the video image, the head and hand images would nolonger be in the video image. If any user-created objects wereassociated with the head image or the hand image, then thoseuser-created objects would be removed because those images would nolonger be in the video image. Additionally, the first user-createdobject may be reinstated and the second user-created object may bereinstated if the first object and the second object reappear in thevideo image. So, in the example described above, if the user reappearsin the video image, then the user-created objects that are associatedwith the head image and the hand image would reappear as well and belocated as before with respect to the objects with which they wereassociated.

The systems and methods provided herein can be implemented in hardware,software, firmware, or a combination thereof. In one embodiment, themethod for generating effects in a webcam application is implemented insoftware or firmware that is stored in a memory and that is executed bya suitable instruction execution system. If implemented in hardware, asin an alternative embodiment, the system for generating effects in awebcam application can be implemented with any or a combination of thefollowing technologies, which are all well known in the art: a discretelogic circuit(s) having logic gates for implementing logic functionsupon data signals, an application specific integrated circuit (ASIC)having appropriate combinational logic gates, a programmable gatearray(s) (PGA), a field programmable gate array (FPGA), etc.

Any process descriptions or blocks in flow charts should be understoodas representing modules, segments, or portions of code which include oneor more executable instructions for implementing specific logicalfunctions or blocks in the process, and alternate implementations areincluded within the scope of the preferred embodiment of the presentinvention in which functions may be executed out of order from thatshown or discussed, including substantially concurrently or in reverseorder, depending on the functionality involved, as would be understoodby those reasonably skilled in the art of the system and method providedherein.

The software program implementing the method for generating effects in awebcam application can comprise an ordered listing of executableinstructions for implementing logical functions, which can be embodiedin any computer-readable medium for use by or in connection with aninstruction execution system, apparatus, or device, such as acomputer-based system, processor-containing system, or other system thatcan fetch the instructions from the instruction execution system,apparatus, or device and execute the instructions. In the context ofthis document, a “computer-readable medium” can be any of a variety ofmeans that can contain, store, communicate, propagate, or transport theprogram for use by or in connection with the instruction executionsystem, apparatus, or device. The computer readable medium can be, forexample but not limited to, an electronic, magnetic, optical,electromagnetic, infrared, or semiconductor system, apparatus, device,or propagation medium. More specific examples (a nonexhaustive list) ofthe computer-readable medium would include the following: an electricalconnection (electronic) having one or more wires, a portable computerdiskette (magnetic), a random access memory (RAM) (electronic), aread-only memory (ROM) (electronic), an erasable programmable read-onlymemory (EPROM or Flash memory) (electronic), an optical fiber (optical),and a portable compact disc read-only memory (CDROM) (optical). Inaddition, the scope of the certain embodiments of the present inventionincludes embodying the functionality of the preferred embodiments of thepresent invention in logic embodied in hardware or software-configuredmediums.

It should be emphasized that the above-described embodiments of thepresent invention are merely possible examples of implementations,merely set forth for a clear understanding of the principles of theinvention. Many variations and modifications may be made to theabove-described embodiment(s) of the invention without departingsubstantially from the spirit and principles of the invention. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and the present invention and protected bythe following claims.

1. A method of generating effects for a webcam application, comprisingthe steps of: identifying a first object in a video image; identifying asecond object in the video image; adding a first user-created object tothe video image to create an altered video image; adding a seconduser-created object to the altered video image to further alter thealtered video image; associating the second user-created object with thesecond object; and identifying a movement of the second object; movingthe second user-created object in the altered video image in accordancewith the association of the second user-created object with the secondobject; and wherein the first object is a static object, wherein thefirst user-created object is manually movable, and wherein the movementof the second user-created object in association with the second objectis independent of a movement of the first user-created object.
 2. Themethod of claim 1, wherein the first object is a background region. 3.The method of claim 1, further comprising associating the firstuser-created object with the first object.
 4. The method of claim 1,wherein the first user-created object and the second user-created objectare selected from a group consisting of: a user-drawn object, auser-selected object, a copied object and text.
 5. The method of claim1, wherein the first user-created object and the second user-createdobject are deformable, respectively.
 6. The method of claim 1, whereinthe first user-created object and the second user-created object areresponsive to a stimulus, respectively.
 7. The method of claim 1,wherein the first user-created object and the second user-created objectmorph temporally, respectively.
 8. The method of claim 1, furthercomprising scaling the second user-created object.
 9. The method ofclaim 8, wherein the scaling of the second user-created object isdependent upon the scale of the second object.
 10. The method of claim1, further comprising shading the second user-created object.
 11. Themethod of claim 10, wherein the shading of the second user-createdobject is dependent upon the shade of the second object.
 12. The methodof claim 1, wherein the identifying movement step further comprisesdefining a second object motion vector based on the movement of thesecond object.
 13. The method of claim 12, further comprising: modifyingthe second user-created object depending on the second object motionvector.
 14. The method of claim 1, wherein the identifying the movementstep further comprises finding difference of the first object and thesecond object using a technique selected from a group consisting offrame comparison analysis, motion detection, object identificationtechnique, or edge detection analysis.
 15. The method of claim 1,further comprising: modifying the first user-created object depending ona user input.
 16. The method of claim 1, further comprising: modifyingthe second user-created object depending on a user input.
 17. The methodof claim 1, further comprising: detecting a collision between the firstuser-created object and the second user-created object, wherein acollision occurs when the first user-created object and the seconduser-created object are at least contiguous.
 18. The method of claim 1,further comprising: detecting an overlapping region where the firstuser-created object is overlapped with the second object; and modifyingthe first user-created object responsive to an overlapping regiondetection.
 19. The method of claim 1, further comprising: tracking thesecond object; removing the second user-created object responsive to adisappearance of the tracked second object from the video image; andreinstating the second user-created object responsive to a reappearanceof the tracked second object in the video image.
 20. A method ofgenerating effects for a webcam application, comprising the steps of:identifying a first object and a second object in a video image; addinga first user-created object to the first object to create an alteredvideo image; adding a second user-created object to the second object tofurther alter the altered video image; associating the firstuser-created object with the first object; associating the seconduser-created object with the second object; identifying a movement ofthe first object and a movement of the second object, respectively; andmoving the first user-created object in accordance with the associationof the first user-created object with the first object.
 21. The methodof claim 20, further comprising: moving the second user-created objectin accordance with the association of the second user-created objectwith the second object, wherein the movement of the second user-createdobject is independent of the movement of the first user-created object.22. The method of claim 21, wherein an effect generated is a sound. 23.The method of claim 21, where in an effect generated is shaking thefirst user-created object.
 24. The method of claim 20, wherein an effectgenerated is deformation of the first user-created object and the seconduser-created object.
 25. The method of claim 20, further comprising:detecting a collision between the first user-created object and thefirst object, wherein a collision occurs when the first user-createdobject and the first object are at least contiguous.
 26. The method ofclaim 20, further comprising: detecting a collision between the firstuser-created object and the second user-created object, wherein acollision occurs when the first user-created object and the seconduser-created object are at least contiguous.
 27. The method of claim 20,further comprising generating an effect in response to detecting acollision.
 28. The method of claim 20, wherein the movement of secondobject is independent of the movement of the first object.
 29. Themethod of claim 20, further comprising: tracking the first object;removing the first user-created object from the altered video imageresponsive to the disappearance of the tracked first object from thevideo image; and reinstating the first user-created object responsive tothe reappearance of the first object in the video image.
 30. The methodof claim 29, further comprising: tracking the second object; removingthe second user-created object responsive to a disappearance of thetracked second object from the video image; and reinstating the seconduser-created object responsive to a reappearance of the tracked secondobject in the video image.
 31. A system for generating effects for awebcam application, comprising: an identification module for identifyinga first object and a second object in a video image; an addition modulefor adding at least one user-created object to the video image to createan altered video image; an association module associating the at leastone user-created object with the first object; a motion detection modulefor identifying a movement of the first object; a movement module formoving the at least one user-created object in accordance with theassociation of the at least one user-created object with the firstobject in the altered video image; and a display module for displayingthe altered video image.
 32. The system of claim 31, further comprising:an effects generation module for generating effects for the at least oneuser-created object.
 33. The system of claim 32, further comprising acollision detection module for detecting a collision between the firstobject and the at least one user-created object and triggering theeffects generation module for generating effects.
 34. The system ofclaim 31, further comprising a database comprising a plurality ofuser-created objects.
 35. The system of claim 31, further comprising auser-drawing module for creating the at least one user-created object.36. The system of claim 31, further comprising a communication systeminterface for sending the altered video image to another communicationsystem interface via the Internet.
 37. The system of claim 31, furthercomprising a tracking module for tracking the first object, removing thefirst user-created object when the tracked first object disappears fromthe video image, and reinstating the first user-created object when thetracked first object reappears in the video image.